Television, film and computer graphics industries usually attempt to make video insertions to a scene of a video sequence as seamless as possible. For example, the virtual insertion should appear to a viewer of the video sequence, to be part of the original scene and the addition of the virtual insertion should not expose any visual artifact unless required by the specification. In broadcast television, automated scene recognition has been used, for instance, to insert virtual advertising and graphic effects into live and recorded broadcasts as described in more detail in, for instance, U.S. Pat. No. 5,264,933 entitled “Television displays having selected inserted indicia” issued to Rosser, et al. on Nov. 23, 1993, the contents of which are hereby incorporated by reference in their entirety.
Most automatic search methodologies employed in the field of adding virtual insertions to a scene in a video sequence take a finite time to execute and often the result of the search may only be available at a later point in time, after one or more frames containing views of substantially the same scene have already been presented to the viewer. This may, for instance, occur after a discontinuity, such as, but not limited to, a cut, a dissolve or a special effect. This may lead to a problem termed “late turn-on” where the virtual insertion may not be inserted in the scene in the first frame or even several subsequent frames after the scene becomes part of the video stream. That is, the virtual insertion will not occur at the so-called scene boundary but will first appear at a later point in time in the video sequence.
This sudden appearance of a virtual insertion into a sequence of similar scenes after not being in an initial view of a similar scene, may be immediately picked up by the human eye, even if there is high motion in the scene, or the view of the scene is being distorted by, for instance, the scene being panned, zoomed or rotated.
In the case of a simple cut transition, such as a stream of video frames in which the scene being displayed switches from one scene to an unrelated scene on subsequent frame, it may be possible to have a sufficiently large delay of the video frames being displayed by the viewer. Even a search algorithm that takes many frames to produce a result may work sufficiently well to have the required insertion components available by the time the frames are to be displayed to the viewer.
Most video streams of interest, however, typically have other transitions besides scene cuts, including, but not limited to, dissolves, fades and graphical transition effects which a simple delay solution may not solve as effectively as broadcast standards require or as viewers expect. Moreover, practical constraints make it typically not feasible, or desirable, to have too great an increase in the pipeline delay of the video. The total processing delay may, for instance, need to be made small and of fixed length because of hardware and cost constraints, and because many other processing subsystems typically run at various fixed stages in the pipeline.
There is, therefore, a continuing need to minimize, or eliminate, any delay in the occurrence of a virtual insertion after a change in scenes in a video stream.